Photometry deals with the measurement of visible light. The human eye can only see light in the visible spectrum and has different sensitivities to light of different wavelengths within the spectrum. When adapted for bright conditions (photopic vision), the eye is most sensitive to greenish-yellow light at 555 nm.
The human eye has three types of color receptor cells (cones) that respond to incident radiation with different spectral response curves. A fourth type of cell (rod) is also present but it plays no role in color vision. The existence of exactly three types of color receptor cells implies that three numerical components are necessary and sufficient to describe a perceived color.
Digital images are comprised of a number of picture elements or pixels. Pixel values in color images are specified by tristimulus values. Tristimulus values are the amounts of three primaries that specify a perceived color. The colors red, green and blue form a preferred set of three primary colors.
A pixel is, generally, the smallest addressable unit of an image on a solid state imager, display screen or in a bitmapped image. They are rated by their number of horizontal and vertical pixels; for example, 1024×768 means 1024 pixels are displayed in each row, and there are 768 rows (lines). Many image-acquisition and display systems are not capable of displaying the different color channels at the same site. This approach is generally resolved by using multiple sub-pixels, each of which handles a single color channel. With color systems, each pixel usually contains values for at least three, e.g., red, green and blue, subpixels.
It is desirable to reduce the average number of physical subpixels contained within one image sensor pixel, and hence image sensor silicon area, while not significantly reducing image quality. The most widespread method to give red, green and blue color sensitivity to image sensors is the application of a color filter mosaic array (CFA) on top of an image sensor. The CFA comprises an array of filters. Each filter limits the wavelengths of light that is provided to their associated image sensor. The most common implementation is the three color red, green, blue (RGB) pattern. Other color implementations exist such as three-color, e.g., yellow, magenta, cyan, complementary patterns or mixed primary/complementary colors and four color systems where the fourth color is a white or a color with shifted spectral sensitivity.
Although many criteria of a technical and physical implementation nature could be applied to choose a CFA pattern some of the most important are immunity to color artifacts and moiré patterns, minimization of pattern interaction with image sensor imperfections, color reconstruction computational complexity and immunity to optical and electrical cross talk between neighboring pixels. Preferred candidate color patterns should therefore have all three, red, green and blue, components available in the neighborhood of each pixel, each pixel should have the same number of neighbors of a given color and diagonal pixel alignment is desirable whenever possible.
FIG. 1 illustrates one of the most common CFA patterns in use today, the Bayer pattern. Each pixel is covered with an individual red, green or blue filter. Thus each image sensor, or pixel, captures only one color; full color values for each pixel are determined by interpolation using surrounding pixel values. Pixels 110 (red), 120 and 130 (green) and 140 (blue) form a basic Bayer pattern that is repeated multiple times to instantiate a practical image sensor. Compared to a monochrome sensor with the same pixel count and dimensions, the CFA approach lowers the available spatial resolution by roughly 30% to 40% and it requires interpolation calculations to reconstruct the color values for each pixel.
While the RGB color model is sufficient for computer graphics rendering of images, it is widely recognized that the YUV color model more accurately models the human perception of color. The YUV color model defines a color space in terms of one luminance (Y) and two chrominance (UV) components (saturation and hue). Digital cameras typically convert the RGB pixel values into YUV components using a picture reconstruction process that approximates the luminance (Y) channel by the green (G) pixels. The luminance for the non-G pixels is approximated by simple interpolation. The chrominance of the red and blue pixels is calculated as Cr=R−Y and Cb=B−Y using the approximate luminance values. The Cr and Cb values are spatially filtered and the missing values are obtained by interpolation. At this time all pixel should have their luminance and chrominance values and their RGB values are computed by R=Y+Cr, B=Y+Cb and G from Y=(R+G+B)/3 or from Y=0.2R+0.7G+0.1B or from some other similar formula. It is apparent from the above that the Bayer CFA approach trades off accuracy and resolution for simplicity.
Although many variants exist on the above method of deriving individual pixel color they all suffer from the fundamental limitation of the Bayer CFA that luminance is not directly available, but rather must be recreated from dispersed green, red and blue chroma information. Because the human eye is most sensitive to luminance information, the images provided using prior art methods are sub-optimal.